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ABSTRACT 

The program, which lasted 136 days, was designed to 
provide classroom or tutorial services in the area of language 
development for pupils served in Chapter 1 eligible facilities for 
the neglected or delinquent. The evaluation of the program was based 
on standardized achievement test information (grades 2-12). Data were 
collected for the period from September 14, 1992 through March 26, 
1993. The Desired Outcome 1 stated that at east 507. of the pupils in 
the sample — those who were English-speaking, and had a valid 
pretes t-posttes t score for Reading Comprehension — would gain at least 
3.0 Normal Curve Equivalent (NCE) points for the instructional 
period. Norm-referenced tests were administered in grades 2-12 in a 
spring-to-spring cycle. Grade 2 pupils received the Metropolitan 
Achievement Tests, Sixth Edition (MAT6, 1985). Pupils in grades J-12 
received the California Achievement Tests (CAT, 1985). Of the 317 
pupi 1 s served , the average days of enrol lment per pupi 1 were 15.1 and 
the average days of attendance per pupil were 11.0. Due to a high 
degree of pupil mobility, there were no pupils who had attended 
enough days for inclusion in the evaluation sample. Therefore no 
assessment could be made based on the program's Desired Outcome. It 
is recommended that the program be continued, since it provides a 
needed service to pupils in exceptional circumstances . It is further 
recommended that alternatives to the present evaluation design be 
considered. Instruments used to collect data are appended. (BF) 
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Elementary and Secondary Education Act - Chapter 1 



FINAL EVALUATION REPORT 
NEGLECTED OR DELINQUENT PROGRAM 
1992-93 

ABSTRACT 



Program Description: The Neglected or Delinquent Program (N or D) is designed to provide 
classroom and tutorial sen/ices in the area of language development for pupils served in Chapter 1 eligible 
facilities for the neglected or delinquent. During the 1992-93 school year, there was one full-time N or D 
reading teacher and 14 part-time tutors providing services in eight institutions. In terms of full-time 
equivalency, the program was served by 3.87 teachers. 

Time Interval: For purposes of evaluating the program based on standardized achievement test 
information (grades 2-12), data were collected for the period from September 14, 1992, through March 26, 
1993. This interval of time provided 136 possible days for instruction. 

Desired Outcome: Desired Outcome 1 (for grades 2-12) stated that at least 50% of the pupils in 
the evaluation sample-those who met the attendance criterion, were English-speaking, and had a valid 
pretest-posttest score for Reading Comprehension-would gain at least 3.0 Normal Curve Equivalent (NCE) 
points for the instructional period. 

Evaluation Design: Norm-referenced tests were administered in grades 2-12 in a spring-to-spring 
cycle. Grade 2 pupils received the Metropolitan Achievement Tests, Sixth Edition (MAT6, 1985). Pupils in 
grades 3-12 received the California Achievement Tests (CAT, 1985). 

Mator Findings: The program served a total of 317 pupils in grades 2-12 for an average of 2.0 
hours of instruction per week. Of the pupils served, 20 were in grades 2 through 5, and 297 were in grades 
6 through 12. The average daily membership was 52.1. The average days of enrollment per pupil was 
1 5.1 and the average days of attendance per pupil was 1 1 .0. There was a high degree of pupil mobility 
due to varying lengths of time pupils were assigned to the institutional facilities served. 

Of the 317 pupils served in the program, 132 were giris and 185 were boys. In regard to ethnic 
origin, 157 pupils were non-minority, 156 were Black, one was Hispanic, and three were Asian American. 
The percent of minority pupils served was 50.5%. 

Due to a high degree of pupil mobility, there were no pupils who had attended enough days for 
inclusion in the evaluation sample. Therefore no assessment could be made based on the program's 
Desired Outcome. 

Recommendations: it is recommended that the N or D Program be continued, since it provides a 
ne ded sen/ice to pupils in exceptional circumstances. It is further recommended that alternatives to the 
present evaluation design be considered. Only a limited number of pupils are enrolled in the program long 
enough to attain the criteria for inclusion in the evaluation sample or treatment group. The value of testing 
under these circumstances would seem questionable. 
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Elementary and Secondary Education Act - Chapter 1 



FINAL EVALUATION REPORT 
NEGLECTED OR DELINQUENT PROGRAM 

1992-1993 



Program Evaluation 

The Neglected or Delinquent Program (N or D) is designed to provide classroom and tutorial services in 
the area of language development for pupils served in Chapter 1 eligible facilities for the neglected or 
delinquent. Ti;e N or D Component became a separate entity under ESEA Title I in the 1968-69 school 
year, at which time emphasis was placed on providing intensive service to pupils residing in institutions. 

During the 1992-93 school year, the program operated in eight institutions and was staffed by one full- 
time N or D reading teacher and 14 part-time tutoring positions. In terms of full-time equivalency, the 
program was served by 3.87 teachers. The term "teachers* will be used in this report to designate 
providers of instruction, whether in a small group or a tutorial setting. The institutions in which services 
occurred were: Rosemont School, Franklin County Juvenile Detention Center, Hampton Group Home, 
Hannah Neil Center, Joyce Aver je Group Home, Karl Road Group Home, Parenthesis Family Advocates, 
and Youth Advocate Services, fhe program served 31 7 pupils in grades 2-1 2. 

Evaluation Design 
The program's Desired Outcome was as follows: 

Desired Outcome 1 : At least 50 percent of the pupils (grades 2-8) in the evaluation sample will 
gain at least 3.0 normal curve equivalent (NCE) points for the instructional period in reading 
comprehension. Gain will be measured by a nationally standardized achievement test. The 
evaluation sample is defined as those pupils who attended the program at least 80 percent of the 
instructional period. 

In addition to data related to reading comprehension as specified in the Desired Outcome, data were 
also collected on total reading. This was done since federal guidelines require that aggregate test data be 
reported for grades 2 and above for both Total Reading and Reading Comprehension for individual 
buildings. For purposes of evaluating achievement test data (grades 2-12), the instructional period was 
considered to be the period from September 14, 1992, through March 26, 1993. This interval of time 
provided 136 possible days for instruction. Since N or D tutoring activities are not bound by the calendar of 
the Columbus Public Schools, the possible days of instruction as stated here include a number of days 
when the Colurrtbus Public Schools was not in session. The maximum possible days of instruction does 
not necessarily define the actual treatment periods. Treatment periods varied according to the service 
patterns of program teachers, which varied from one day a week through five days a week. 

Instruments 

Data were collected in three areas. Copies of the instruments used to collect data are found in the 
Appendix, with the exception of the standardized achievement tests. 
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1 . Pupil Census Instalments 

Calendar Worksheet for Recording Davs of Pupil Service, T.ie Calendar Woifcsheet was used to help 
program teachers collect program service data. A Calendar Worksheet was kept for each pupil. The 
form included the following information: the pupil's name, birthdate, student number, ethnic or race 
code, sex, grade, and hours of instruction scheduled per week. Days of instruction scheduled and 
served were recorded by program teachers so that correct information was available to report at the 
end of the year on the Pupil Data Sheet. Copies of the Calendar Worksheets were collected at regular 
intervals in order to maintain a master file of Chapter 1 pupils served in facilities other than those 
provided by the school system. A copy of the Calendar Worksheet is found in the Appendix, page 1 1 . 

Pupil Data Sheet. The Pupil Data Sheet was developed to help program teachers summarize the pupil 
information from the Calendar Worksheet at the end of the school year. This instrument was used to 
collect the following information: data identifying those pupils who were non-English speaking, 
subjective ratings of pupil progress given by teachers, the number of hours of instruction per week, 
number of days scheduled for instruction, number of days sen/ice received, and the service pattern 
(average number of days scheduled per week for instruction). Days scheduled and days of instruction 
were specified for the period from September 14, 1992 through March 26, 1993. The attendance 
criterion for the Desired Outcome was applied to this time period, taking into account the teacher's 
service pattern. A copy of the Pupil Data Sheet is found in the Appendix, page 1 2. 

2. Standardized Achievement Test Instruments 

Metropolitan Achievement Tests Sixth Edition (MAT6. 1985). Second-grade pupils were administered 
the Metropolitan Achievement Tests, Sixth edition (MAT6, 1985), which is published by the 
Psychological Corporation/Harcourt Brace Jovanovich, Inc. This test series has two sets of norms 
(national and nonpublic) for fall and spring. Standardization was established between October 1 and 
31 in 1984 for fall, and spring standardization was established between April 8 and May 15 in 1985. 

California Achievement Tests (CAT, 1985). The California Achievement Tests were administered to 
program pupils in grades 3-12. This test series, wh^ch is published by CTB/McGraw-Hill, has empirical 
norms for fall and spring, established in the fall of 1984 and the spring of 1985. 

All testing was done on level using the Norm-Referenced Model for evaluation of the Chapter 1 N or D 
Program. A spring-to-spring testing cycle was used for grades 2-12. The form, subtest, and test levels 
used for each grade level are shown in Table 1 . Many pupils served in institutions for the neglected 
and delinquent also attended schools in the Columbus Public Schools. Where this was the case pupils 
in grades 2-8 were tested in their regular classrooms as part of Districtwide Testing. Pupils in grades 9- 
12, and pupils not attending a Columbus Public School, were tested by their program teachers. 

3. Inservice Evaluation Instrument 

Orientation Inservice Evaluation Form. This locally developed instrument was designed to obtain 
teacher perceptions regarding the Orientation inservice session, which was held September 10, 1992. 
An abbreviated version of the instrument, the General Inservice Evaluation Form, was used at a 
Chapter 1 inservice meeting regarding use of the Instructional Handbook, which was held October 15, 
1992, and was attended by one of the N or D teachers. The two instruments used for inservice 
evaluation are found in the Appendix, pages 13-15. While the evaluation design does not provide for 
the collection of these data (nor are the findings reported here), interim inservice evaluation reports for 
the two meetings were forwarded to Federal and State Programs, where they are available on request. 
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Major Findings 

Pupil Census Information 

A total of 317 pupils in institutions was served by the ESEA Chapter 1 N or D Program during the 1992- 
93 school year for an average of 2.0 hours of instruction per week. Of the pupils served, 20 were in grades 
2 through 5, and 297 were in grades 6 through 12. The average daily membership in the program was 
52.1. The average days of enrollment per pupil was 15.1 and the average days of attendance per pupil 
was 1 1 .0. Data pertaining to enrollment and attendance by grade are presented in Table 2. 

Of the 317 pupils served in the program, 132 were girls and 185 were boys. In regard to ethnic origin, 
1 57 pupils were non-minority, 1 56 were Black, one Hispanic, and three wer^ Asian American. The percent 
of minority pupils served was 50.5%. Table 3 contains data on ethnic origin of pupils served by grade. 

Pupils were served in eight institutions by one full-time N or D reading teacher and 14 tutors working on 
a part-time basis. Both full-time and tutorial instructors are referred to as teachers in this report. In terms of 
full-time equivalency the program was served by 3.87 teachers. The average number of pupils served by 
each of the 1 5 teachers during the school year was 21.1, with the average number of pupils per teacher at 
any given time being 3.5 based on average daily membership. The pupil-teacher ratio based on average 
daily membership divided by the teacher full-time equivalency of 3.87 was 13.5. There was a high degree 
of pupil mobility due to the varying lengths of time pupils were assigned to the institutional facilities served. 

The service patterns (average days of service per week) varied by teacher, ranging from one to five 
days per week. Of the fifteen teachers, two (13.3%) provided service five days a week, three (20.0%) 
served four days a week, three (20.0%) served three days a week, six (40.0%) served two days a week, 
and one (6.7%) served one day a week. 

Pupil census information also included teacher subjective ratings of pupil progress as pupils exited the 
program. Of the 317 pupils served in the program, 49 (15.5%) were rated by their prog; am teachers as 
making much progress, 222 (70.0%) as making some progress, and 46 (14.5) as making no progress. For 
progress ratings by grade see Table 4. 

Standardized Achievement Test Information 

Standardized Achievement test data were of *wo types. Reading Comprehension scores were used to 
assess Desired Outcome 1: that at least 50 percent of the pupils (grades 2-12) in the evaluation sample 
will gain at least 3.0 NCE points for the instructional period in Reading Comprehension. In addition, 
aggregate scores were obtained for both Total Reading and Reading Comprehension. Federal guidelines 
require that aggregate test data be reported for grade 2 and above for both Total Reading (Basic Skills) and 
Reading Comprehension (Advanced Skills) for individual buildings. 

In order to be in the evaluation sample pupils had to be English-speaking, meet the attendance criterion 
(80% of the days scheduled for instruction) and have both a valid pretest and a posttest score. In addition, 
pupils who were eligible for Special Education were exempted from testing since their test scores would not 
be considered valid. 

Of the 317 pupils served in the program, all but one were English-speaking. The number of pupils 
exempted from testing due to their Special Education status was 1 1 6. However in this year's program there 
were no pupils who met the attendance criterion for inclusion in the evaluation sample, regardless of 
English-speaking or Special Education status. There were matched scores (pretest and posttest) for only 
14 pupils in Reading Comprehension, and for only 13 pupils in Total Reading. However, these pupils did 
not meet the attendance criterion for inclusion in the evaluation sample. Therefore, the program could not 
be evaluated in terms of norm-referenced test data. 
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Table 2 

Number of Pupils Served and Averages for Days of Enrollment, Days 
of Attendance, Daily Membership and Hours of Instruction Per 
Week Reported by Grade Level for N or D Program 
1992-93 



Average 



Grade 


N 


Pupils Served 
Girls Boys 


Days of 
Enrollment 


Days of 
Attendance 


Daily 
Membership 


Hours of Instruction 
per Pupil per Week 


2 


6 


1 


5 


36.5 


30.3 


1.6 


1.4 


3 


4 


i 

i 


3 


38.8 


36.0 


1.1 


1.4 


4 


5 


0 


5 


41.0 


32.2 


1.5 


1.5 


5 


5 


1 


4 


21.4 


19.6 


1.2 


2.2 


6 


18 


6 


12 


8.9 


8.7 


2.2 


1.7 


7 


51 


24 


27 


15.7 


9.6 


7.5 


2.2 


8 


73 


32 


41 


19.7 


12.0 


14.8 


2.0 


9 


73 


31 


42 


11.8 


9.7 


11.7 


2.2 


10 


50 


22 


28 


8.4 


6.3 


5.6 


1.6 


11 


27 


10 


17 


9.0 


6.9 


3.3 


2.2 


12 


5 


4 


1 


38.6 


34.2 


1.5 


2.8 



Total 317 132 185 15.1 ]±0 5Z1 2J) 
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Table 3 

Number of Pupils Served by Ethnic Origin and 
Percent of Minority Pupils Reported by 
Grade Level for N or D Program 
1992-93 



Grade 


Pupils 
Served 


Non-Minority 


Black 


Spanish 


Asian 
American 


American 
Indian 


Percent 
Minority 
Pupils 


2 


6 


5 


1 


0 


0 


0 


16.7 


3 


4 


1 


3 


0 


0 


0 


75.0 


4 


5 


4 


1 


0 


0 


0 


20.0 


5 


5 


1 


4 


0 


0 


0 


80.0 


6 


18 


10 


8 


o 


0 


0 


44.4 


7 


51 


26 


25 


0 


0 


0 


49.0 


8 


73 


44 


28 


0 


1 


0 


39.7 


9 


73 


34 


39 


0 


0 


0 


53.4 


10 


50 


20 


27 


1 


2 


0 


60.0 


11 


27 


10 


17 


0 


0 


0 


63.0 


12 


5 


2 


3 


0 


0 


0 


60.0 


Total 


317 


157 


156 


1 


3 


0 


50.5 
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Table 4 

Progress of N or D Program Pupils as Rated 
by Program Teachers 
by Grade Level 
1992-93 



Descriptors of Amount of Progress 



vjraae 


rupilS 

Served 


Much 


Some 


None 


N 


% 


N 


% 


N 


% 


2 


O 


2 


33.3 


3 


50.0 


1 


1 b. i 


3 


A 

4 


2 


50.0 


1 


25.0 


1 


25.0 


4 


5 


1 


20.0 


2 


40.0 


2 


40.0 


5 


5 


.4 


80.0 


1 


20.0 


0 


0.0 


6 


18 


1 


5.6 


15 


83.3 


2 


11.1 


7 


51 


8 


15.7 


31 


60.8 


12 


23.5 


8 


73 


7 


9.6 


54 


74.0 


12 


16.4 


9 


73 


13 


17.8 


51 


69.9 


9 


12.3 


10 


50 


4 


8.0 


44 


88.0 


2 


4.0 


11 


27 


4 


14.8 


19 


70.4 


4 


14.8 


12 


5 


3 


60.0 


1 


20.0 


1 


20.0 


Total 


317 


49 


15.5 


222 


70.0 


46 


14.5 
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Summary/Recommendations 



The Chapter 1 Neglected or Delinquent Program (N or D) is designed to provide classroom and tutorial 
instructional services in language development for pupils served in Chapter 1 eligible facilities for the 
neglected or delinquent. For puiposes of evaluating the program based on standardized achievement test 
information, data were collected from September 14, 1992, through March 26, 1993. This interval of time 
provided 136 possible days oi instruction. 

The program provided instructional services in language development to 317 pupils in grades 2-12, 
served by 1 5 teachers in eight institutions for the neglected or delinquent. Ail but one teacher served part- 
time, and the fuil-time equivalency was 3.78 teachers. Teachers 1 service patterns ranged from one day a 
week to five days a week. The average daily membership was 52.1 . There was considerable turnover in 
pupil enrollment during the year, as is normal in N or D facilities. The average days of enrollment per pupil 
was 1 5.1 days, and the average days of attendance was 1 1 .0. 

The Desired Outcome for grades 2-12 stated that at least 50 percent of the pupils in the evaluation 
sample (those pupils who had a valid pretest and posttest measure, attended the program at least 80 
percent of the instructional period, and were English-speaking) would gain at least 3.0 normal curve 
equivalent (NCE) points for the instructional period in Reading Comprehension, as measured by a 
nationally standardized achievement test. One pupil was non-English speaking and 116 pupils were 
exempted from testing due to their Special Education Status. However, none of the pupils met the 
attendance criterion regardless of English-speaking or Special Education status. Therefore there was no 
evaluation sample for this Desired Outcome. 

It is recommended that the N or D Program be continued, since it provides a needed service to pupils 
in exceptional circumstances. It is further recommenced that alternatives to the present evaluation design 
be considered. Only a limited number of pupils are enrolled in the program long enough to satisfy 
attendance criteria for inclusion in the evaluation sample or treatment group. In addition, many pupils are 
exempt from testing due to their special education status. The value of testing under these circumstances 
would seem questionable. 
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: c : umous ^'jD 1 i c S zr.co >. s 
::moensatorv icucaticn ^^cgrams 

= UPIL 3 A T A SHEET 

PROGRAM CwCE 3 3 0 6 C SSN _ 



?93 



:hoc'w name 

student name 
<. . student no. 



PROGRAM NAME 



r EACHER NAME 



as t 

3RAOE 



3.-;TH0ATE 



first 

/ / 



3. AVERAGE HOURS PER WEEK CF INSTRUCTION 
a . PUPIL PROGRESS 

5. IS THIS PUPIL ENGLISH SPE AKING7 



NONE 
NO 



SOME 
YES 



MUCH 



i. NUMBER OF DAYS SERVICE SCHEDULED 
(CAREFULLY READ INSTRUCTIONS) 



THRU 03-26-93 



! I I 



7. NUMBER OF DAYS SERVICE RECEIVED 
(CAREFULLY READ INSTRUCTIONS) 



8. CN AVERAGE, "HIS PUPIL WAS 
SCHEOULED TO RECEIVE SERVICE 



DAYS OUT OF 



16 



prtoina Dr- 
aff ice of tne Superintendent 
Department of Prcgrem Evaluation (pif pas) 



i: 



ESEA CHAPTER 1 AND DPPF 
ORIENTATION INSERVICE EVALUATION FORM 
1992-93 ORIENTATION 



Date of Orientation Meeting 

Circle cnly.the program(s) you are in: 

ESEA Chapter 1 Programs: 

(1) Reading Elementary (1-5) 

(2) Mathematics-Elementary (3-5) 

(3) Reading-Middle School (6-8) 

(4) Mathematics-Middle School (6-8) 

(5) N or D (1 12) 

(6) Nonpublic (1-8) 

(7) Reading Recovery (1) 

(8) Early Literacy (1-2) 



A.M. 



P.M. ALL DAY 



DPPF Programs: 

(9) Instructional Assistant - K 

(10) ADK 

(11) Early Literacy - 2 

General Fund Program: 

(12) HSCA/SSS 

Other (Specify) 

(13) 



Circle the number that indicates the extent to which you agree with statements 1 -4, in rating the overall day 
of inservice. 



Strongly 
Agree 



1 . I think this was a very worthwhile 
inservice. 

2. The information presented in this 
inservice will assist me in my program. 

3. There was time to ask questions 
pertaining to the presentations. 

4. Questions were answered adequately. 



Agree 
4 



Undecided 
3 



Disagree 
2 



Strongly 
Disagree 

1 



5 
5 



4 
4 



3 
3 



2 
2 



1 



Circle the number that indicates how you would rate each of the following portions of today's inservice in 
reqari. to interest and usefulness of presentations. 



Superior Excellent Good 



5. Program Coordinators' Presentation 

a. Interest 

b. Usefulness 

c. Clarity of instructions 

6. Evaluation Presentation 

a. Interest 

b. Usefulness 

c. Clarity of instructions 



5 
5 
5 

5 
5 
5 



4 
4 



4 
4 



3 
3 

3 
3 



Fair 

2 
2 
2 

2 
2 
2 



Poor 



Please turn over for questions 7»9 
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7. What was the most valuable part of this meeting?^ 



8. What was the least valuable part of this meeting? 



9. What additional information or topics would you like to see covered in future meetings? 



is 
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GENERAL INSERVICE EVALUATION FORM 
1992-93 



Inservice Topic: 

Presenter(s): 

Date: / / (e.g., 03/05/93) 
MM DD YY 

Session (Check only one): all day 

Circle only the program(s) you are in: 

ESEA Chapter 1 Programs: 

(1) Reading-Elementary (1-5) 

(2) Mathematics-Elementary (3-5) 

(3) Reading-Middle School (6-8) 

(4) Mathematics-Middle School (6-8) 

(5) N or D (1-12) 

(6) Nonpublic (1-8) 

(7) Reading Recovery (1) 

(8) Early Literacy (1-2) 



a.m. p.m. after school 



DPPF Programs: 

( 9) Instructional Assistant - K 

(10) ADK 

(11) Early Literacy - 2 

General Fund Program: 

(12) HSCA/SSS 

Other (Specify) 

(13) 



Circle the number that indicates the extent to which you agree or disagree with statements 1 -4. 



Strongly Strongly 

Agree Agree Undecided Disagree Disagree 

1 . I think this was a very worthwhile inservice. 5 4 3 2 1 

2. The information presented in this inservice 

will assist me in my program. 5 4 3 2 1 

3. There was time to ask questions pertaining 

to the presentation. 5 4 3 2 1 

4. Questions were answered adequately. 5 4 3 2 1 

5. What was the most valuable part of this meeting? 



6. What was the least valuable part of this meeting? 



7. Please list any additional information or topics you would like to see covered in future meetings. 



b) 
c) 
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